[% setvar title Arrays: Apply operators element-wise in a list context %]
<div id="archive-notice">
    <h3>This file is part of the Perl 6 Archive</h3>
    <p>To see what is currently happening visit <a href="http://www.perl6.org/">http://www.perl6.org/</a></p>
</div>
<div class='pod'>
<a name='TITLE'></a><h1>TITLE</h1>
<p>Arrays: Apply operators element-wise in a list context</p>
<a name='VERSION'></a><h1>VERSION</h1>
<pre>  Maintainer: Jeremy Howard &lt;<a href='mailto:j@howard.fm'>j@howard.fm</a>&gt;
  Date: 10 Aug 2000
  Last Modified: 21 Sep 2000
  Mailing List: <a href='mailto:perl6-language-data@perl.org'>perl6-language-data@perl.org</a>
  Number: 82
  Version: 4
  Status: Frozen</pre>
<a name='DISCUSSION'></a><h1>DISCUSSION</h1>
<p>The first source of discussion was around whether there is any consistent
meaning to array operations. The source of this was that some felt that
other people may want <code>*</code> to mean something other than element-wise
multiplication by default (e.g. matrix inner product). However no-one
actually said that <i>they</i> wanted it to work this way, only that <i>others</i>
may prefer it, particularly mathematicians. The standard use of
element-wise operations in mathematical programming languages such as
Mathematica and J suggests that this is unlikely to be a source of
confusion in practice.</p>
<p>The second source of discussion was around whether <code>||</code> and <code>&amp;&amp;</code> should
be an exception, as specified in RFC 45. This is discussed in detail in
the CONFLICTS section.</p>
<a name='ABSTRACT'></a><h1>ABSTRACT</h1>
<p>It is proposed that in a list context, operators are applied element-wise
to their arguments. Furthermore, it is proposed that this behaviour be
extended to functions that do not provide a specific list context.</p>
<a name='CHANGES'></a><h1>CHANGES</h1>
<a name='Since v3'></a><h2>Since v3</h2>
<ul>
<li><a name='Added discussion of conflict with RFC 45'></a>Added discussion of conflict with RFC 45</li>
</ul>
<a name='Since v2'></a><h2>Since v2</h2>
<ul>
<li><a name='Extended to work with multidimensional arrays'></a>Extended to work with multidimensional arrays</li>
<li><a name='Added ability to broadcast vectors across multidimensional arrays'></a>Added ability to broadcast vectors across multidimensional arrays</li>
<li><a name='Made operating on non-equal sized lists an error'></a>Made operating on non-equal sized lists an error</li>
</ul>
<a name='Since v1'></a><h2>Since v1</h2>
<ul>
<li><a name='Added the ability to apply an operator to a scalar and a list.'></a>Added the ability to apply an operator to a scalar and a list.</li>
<li><a name='Added more examples, including text processing examples.'></a>Added more examples, including text processing examples.</li>
</ul>
<a name='DESCRIPTION'></a><h1>DESCRIPTION</h1>
<p>Currently, operators applied to lists in a list context behave
counter-intuitively:</p>
<pre>  @b = (1,2,3);
  @c = (2,4,6);
  @d = @b * @c;   # Returns (9) == scalar @b * scalar @c</pre>
<p>This RFC proposes that operators in a list context should be applied
element-wise to the elements of their arguments:</p>
<pre>  @d = @b * @c;   # Returns (2,8,18)</pre>
<p>If the lists are not of equal length, an error is raised.</p>
<a name='Multidimensional array operations'></a><h2>Multidimensional array operations</h2>
<p>RFC 202 describes multidimensional arrays in Perl 6. Element-wise list
operations also apply to multidimensional arrays:</p>
<pre>  my int @mat1 = ([1,2],
                  [3,4]);
  my int @mat2 = ([2,2],
                  [1,1]);
  my @mat3 = @mat1 * @mat2;   # ([2,4],[3,4])
  </pre>
<p>An error is raised if the two arrays do not have equal dimensions.</p>
<a name='Broadcasting'></a><h2>Broadcasting</h2>
<p>If an operator is used in a list context with one list (or
multidimensional array), and one or more scalars, the scalars are
treated as if they were an array of that scalar with the same dimensions
as the array:</p>
<pre>  my int @mat1 = ([1,2],
                  [3,4]);
  @e = @mat1 * 2;      # ([2,4],[6,8])
  @f = @mat1 * (2,2,2) # Same thing</pre>
<p>If one operand is a vector of the same bounds as the equivalent dimension
of the other operand, the vector's elements are 'broadcast' across every
other dimension of the other operand:</p>
<pre>  my int @mat1 = ([1,2],
                  [3,4]);
  my int @vec1 = (2,3);     # 1st dimension
  @g = @mat1 * @vec1;       # ([2,4],[9,16])
  my int @vec2 = ([2],[3]); # 2nd dimension
  @h = @mat1 * @vec2;       # ([2,6],[6,12])</pre>
<p>If the operands are a column vector and a row vector, the elements of each
vector are combined into a two dimensional array:</p>
<pre>  my int @vec1 = (2,3);     # 1st dimension
  my int @vec2 = ([2],[3]); # 2nd dimension
  @i = @vec1*@vec2;         # ([2*2,3*2],[2*3,3*3]) == ([4,6],[6,9])</pre>
<p>Equivalent combinatorial broadcasting occurs if the operands are
perpendicular planes (creating a cube), and so forth for higher
dimensional arrays.</p>
<a name='Element-wise functions'></a><h2>Element-wise functions</h2>
<p>Functions that do not return a list should be treated in the same way:</p>
<pre>  @e = (-1,1,-3);
  @f = abs(@e);   # Returns (1,1,3)</pre>
<a name='EXAMPLES'></a><h1>EXAMPLES</h1>
<a name='Text processing'></a><h2>Text processing</h2>
<p>If @first_names contains a list of peoples first names, and @surnames
contains their surnames, this creates a new list that concatenates the
elements of the two lists:</p>
<pre>  @full_names = @first_names . @surnames;</pre>
<p>To quote a number of lines of a message by prefixing them all with '&gt; ':</p>
<pre>  @quoted_lines = '&gt; ' . @raw_lines;</pre>
<p>To create a histogram for a list of scores:</p>
<pre>  @people = ('adam', 'eve ', 'bob ');
  @scores = (7,9,5);          # Score for each person
  @histogram = '#' x @scores; # Returns ('xxxxxxx','xxxxxxxxx','xxxxx')
  print join(&quot;\n&quot;, @people . ' ' . @histogram);
  
  adam xxxxxxx
  eve  xxxxxxxxx
  bob  xxxxx
  </pre>
<a name='Number crunching'></a><h2>Number crunching</h2>
<p>This snippet multiplies the absolute values of three arrays together and
sums the results, in a very efficient way:</p>
<pre>  @b = (1,2,3);
  @c = (2,4,6);
  @d = (-2,-4,-6);
  $sum = reduce ^_+^_, abs(@b * @c + @d);</pre>
<p>Lists can be reordered or sliced with list generation functions (RFC 81)
allowing flexible data manipulation:</p>
<pre>  @a = (3,6,9);
  @reverse = (3..1);
  @b = @a * @a[@rev];   # (3*9, 6*6, 9*3) = (27,36,27)</pre>
<p>Slicing plus array operations makes matrix algebra easy:</p>
<pre>  @a = (1,2,3,
        2,4,6,
        3,6,9);
  @column1of3 = (1..7:3);   # (1,4,7) - every 3rd elem from 1 to 7
  @row1of3 = (1..3);       # (1,2,3)
  $sum_col1_by_row1 = 
    sum ( @a[@column1of3] * @a[@row1of3] );  # (1*1+2*2+3*3)=14
  </pre>
<a name='CONFLICTS'></a><h1>CONFLICTS</h1>
<p><i><a href='#RFC 45'>&quot;RFC 45&quot;</a></i> specifies alternative semantics for <code>&amp;&amp;</code> and <code>||</code> in a list
context, which is to evaluate the operands in a scalar context and then
propagate the result to the left-hand side in a list context. The authors
have been unable to resolve this conflict.</p>
<p>This author does not support RFC 45, partly because it breaks consistency
with RFC 82, and partly because it makes || and &amp;&amp; act in a weird cross
between scalar and list context (evaluates in scalar context, propagates
in list context).</p>
<p>Also, boolean operations are frequently applied to lists of elements.
There is really nothing about this operation that makes it only useful for
scalars. For example:</p>
<pre>  @mask = (1,0,1,0);
  @a = (4,3,2,1);
  @b = ('a','b','c','d');

  @mult_a = @mask * @a;   # (4,0,2,0)
  @mask_a = @mask &amp;&amp; @a;  # (4,0,2,0)
  @mask_b = @mask &amp;&amp; @b;  # ('a',0,'c',0)</pre>
<p>Finally, the control-flow/short-circuiting side effet of || and &amp;&amp; would
be very useful for array operations. In lazily generated lists, using ||
or &amp;&amp; for an element-wise operation would avoid computing elements of the
RHS where not necessary to do so. RFC 45's proposal really only talks
about flow-control (rather than boolean operations), but flow control
statements can be used directly:</p>
<pre>  @a = @c unless @a = @b;</pre>
<p>achieves the same thing as RFC 45's proposed</p>
<pre>  @a = @b || @c</pre>
<p>without the double evaluation of @b caused by</p>
<pre>  @a = @b ? @b : @c</pre>
<a name='IMPLEMENTATION'></a><h1>IMPLEMENTATION</h1>
<p>These operators and functions should be evaluated lazily. For instance:</p>
<pre>  @b = (1,2,3);
  @c = (2,4,6);
  @d = (-2,-4,-6);
  $sum = reduce ^_+^_, @b * @c + @d;
  </pre>
<p>should be evaluated as if it read:</p>
<pre>  $sum = 0;
  $sum += $b[$_] * $c[$_] + $d[_] for (0..$#a-1));</pre>
<p>That is, no temporary list is created, and only one loop is required.</p>
<p>The proposal to handle functions is tricky, since there is currently no
obvious way to see whether a function is going to return a list. For
instance, in the case:</p>
<pre>  @b = abs(@a);
  </pre>
<p>we either need some kind of more advanced prototyping (or other way of
creating a signature) so that Perl knows to apply abs() to the elements of
@a, or we need to manually change abs to check for list context and Do The
Right Thing.</p>
<a name='REFERENCES'></a><h1>REFERENCES</h1>
<p>The Mathematica Navigator, Heikki Ruskeepää, Academic Press, ISBN
0-12-603640-3, p383.</p>
<p>Expression Templates (C++ Implementation): <a href='http://extreme.indiana.edu/~tv-' target='_blank'>extreme.indiana.edu</a>
eldhui/papers/techniques/techniques01.html#l32</p>
<p>Implementation in Perl Data Language: <a href='http://pdl.perl.org' target='_blank'>pdl.perl.org</a></p>
<p>Array operations in Numerical Python:
<a href='http://starship.python.net/~da/numtut/array.html' target='_blank'>starship.python.net</a>#SEC10</p>
<p>RFC 76: reduce</p>
<p>RFC 23: Higher-order functions</p>
<p>RFC 81: Lazily evaluated list generation functions</p>
<p>RFC 202: Overview of multidimensional array RFCs</p>
</div>
